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Abstract 
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1 Introduction 

It is broadly accepted that interior point methods (IPMs) provide very efficient solution tech- 
niques for linear and convex quadratic programming problems 12 lj . Interior point algo- 
rithms for these classes of problems enjoy excellent worst-case complexity bounds: indeed the 
best known algorithms find the e-accurate optimal solutions to the problem with n variables 
in 0{^/n\og{l/e)) or 0(n log(l/e)) iterations, depending on how aggressive steps to optimality 
are allowed. Computational experience provides evidence that the algorithm which uses a more 
aggressive strategy (the so-called long-step method) solves linear and quadratic programming 
problems in a number of iterations which may be expressed as C(lognlog(l/e)) 

The small number of iterations does not always guarantee the efficiency of the method because 
occasionally IPMs struggle with a high per-iteration cost of the linear algebra operations. In 
the most adverse case the cost of solving a dense optimization problem employing a direct linear 
algebra method to solve the Newton equation system may reach C(n 3 ) flops per iteration. The 
effort of a single IPM iteration is usually significantly lower than this upper bound. However if 
problems are very large then, although they may display reasonable sparsity features, the use 
of direct sparsity-exploiting linear algebra techniques may still run into trouble due to excessive 
memory requirements or unacceptably long CPU time. Iterative methods for linear equations 
such as conjugate gradients or other approaches from the Krylov subspace family may offer a 
viable alternative to direct methods in such cases. 

The interest in the use of iterative methods to solve the Newton equation system in IPMs has 
been growing over the last decade: see for example the recent survey of D'Apuzzo et al. [8]. 
Iterative methods offer several advantages. In particular they are often memory efficient and 
hence allow much larger problems to be solved. However to take full advantage of iterative 
methods one usually has to relax the accuracy requirements in the solution of the Newton 
equation system. Consequently, instead of using an exact Newton direction, the resulting IPM 
employs an inexact one. This opens up all sorts of theoretical and practical questions. We 
will state and answer some of these in this paper. In particular, we will discuss the key issue 
concerning an acceptable level of error in the inexact Newton method used by an IPM. 

The use of an inexact Newton method [9] is well established in the context of solving nonlinear 
equations |13j and in nonlinear optimization |18j . Bellavia [5 J applied an inexact interior point 
method to solve monotone nonlinear complementarity problems and proved global convergence 
and local superlinear convergence of the method. Several successful attempts were also made 
to shed light on the application of an inexact Newton method in IPMs for linear and convex 
quadratic programming problems. 

Freund, Jarre and Mizuno [10] and Mizuno and Jarre |17j extended a very popular globally 
convergent infeasible path- following method for linear programming of Kojima, Megiddo and 
Mizuno [H] to accommodate the inexact solution of Newton systems. In particular Mizuno 
and Jarre |17j proved that an inexact variant of this algorithm has 0(n 2 log(l/e)) complexity. 
Baryamureeba and Steihaug [3] provided another extension of the method of Kojima et al., 
allowing for inexactness in both primal and dual Newton steps. All these analyses considered 
a general case in which no assumption was made about how the Newton system is solved. The 
only assumptions made were concerned with the absolute or relative error in the inexact Newton 
direction. 
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Several authors tried to specialize their analyses using a better understanding of the specific 
iterative methods of linear algebra employed to solve the Newton equation system inexactly. 
Indeed, if a particular preconditioner is used in a given Krylov subspace method applied in 
this context, it is often possible to control the residuals in specific linear subspaces and design 
special variants of an inexact interior point algorithm. Al-Jeiroudi and Gondzio [1] considered the 
case in which an indefinite preconditioner based on a guess of basic-nonbasic partition inspired 
by the work of Oliveira and Sorensen [19] is used to precondition the indefinite augmented 
system (the reduced KKT system). They designed an inexact infeasible primal-dual IPM for 
linear programming and established its 0(n 2 log(l/e)) complexity. Lu, Monteiro and O'Neal [15] 
analysed the case of quadratic programming in which the matrix of the quadratic objective term 
has a known factorization and proposed an interesting specialized preconditioner for the Newton 
system in this case. They showed that the resulting inexact IPM converges in 0(n 2 log(l/e)) 
iterations. Cafieri et al. [7} performed an analysis of an inexact potential reduction algorithm 
for convex quadratic programming. 

In this paper we consider a feasible primal-dual path-following method for convex quadratic 
programming and analyse it in a situation when the solutions of the Newton systems admit a 
certain level of inaccuracy. We prove worst-case iteration complexity results for two variants of 
such a method. The first variant requires the iterates to stay in a small neighbourhood of the 
central path induced by the use of the Euclidean norm to control the error in the perturbed 
complementarity conditions. Such a method has the best known iteration complexity result 
0(y/n log(l/e)) and we prove that the inexact variant preserves the same complexity. However, 
this method is only of theoretical interest because its implementation demonstrates the behaviour 
predicted by the worst-case analysis. Therefore, this algorithm is not used in practice. The 
second variant allows the iterates to stay in a wide symmetric neighbourhood of the central path 
induced by the use of the infinity norm to control the error in the perturbed complementarity 
conditions. We show that this method reaches an e-optimal solution in 0(nlog(l/e)) iterations. 
The second algorithm has a practical meaning and although its worst-case iteration complexity 
is y/n times higher than the former one, in practice this algorithm behaves very well and provides 
the basis of an implementable method. 

To simplify the analysis and to allow the reader to concentrate on the essential consequences of 
inexactness in the Newton direction we will analyse feasible algorithms. It is worth mentioning 
that this does not limit the applicability of the analysis. Indeed, the homogeneous and self-dual 
emdedding [22] provided an elegant tool to transform the linear programs into new models for 
which the primal and dual initial feasible solutions are known and therefore primal and dual 
feasibility can be maintained throughout the computations. The homogeneous and self-dual 
emdedding was initially used in the context of LPs and implemented by Andersen and Andersen 
[2] in their Mosek software. Later this elegant model was extended to conic optimization by 
Andersen et al. 

Finally, let us comment that the key motivation for this work is the need to better understand 
how much inaccuracy is admissible in the Newton systems and provide the foundations for 
cases in which interior point algorithms rely on iterative methods to solve the underlying linear 
algebra problems. There has recently been a shift of interest in the IPM community towards the 
application of iterative methods to solve the reduced KKT systems [8]. There exists a rich body 
of literature (cf. Benzi et al. [6]) which deals with very similar saddle point problem arising 
in the discretisations of partial differential equations and we expect that the coming years will 
bring many interesting developments in iterative methods applicable to IPMs. In particular 
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there is a clear need to develop alternative preconditioners which are compatible with the spirit 
of the matrix- free interior point method [11[ I12j . 

The paper is organised as follows. In Section [2] we will introduce the quadratic optimization 
problem, define the notation used in the paper and point out an essential difference between 
the exact and inexact interior point methods. In Section [3] we will perform the worst-case 
analysis of two variants of an inexact feasible interior point algorithm for convex quadratic 
programming. First we will analyse the algorithm operating in a small neighbourhood of the 
central path induced by the 2-norm. Such a method yields the best complexity result known 
to date but it has only a theoretical importance. Next we will analyse the feasible algorithm 
operating in a symmetric neighbourhood of the central path induced by the infinity norm. This 
method has a practical meaning as an implementable algorithm. It provides a theoretical basis 
for the recently developed matrix-free variant of the interior point method [12]. Our analysis 
will follow that of Wright [21] and will generalize it from linear programming to quadratic 
programming and from exact to inexact method. In Section 2] we will briefly comment on some 
practical aspects related to the implementation of inexact interior point method and finally in 
Section [5] we will give our conclusions. 



2 Interior point methods: background 

We are concerned in this paper with the theory of interior point methods for solving convex 
quadratic programming (QP) problems. We consider the following general primal-dual pair of 
QPs 

Primal Dual 

min c x + ^x Qx max by — -^x Qx m 

s.t. Ax = b, s.t. A T y + s — Qx = c, 

x > 0, y free, s > 0, 

where A G -j^/nxn f u jj row ran k m < n, Q G TZ nxn is a positive semidefinite matrix, 
x,s,c G 1Z n and y,b G 1Z m . Setting Q = yields the special case of the linear programming 
(LP) primal-dual pair. 

To derive a primal-dual interior point method [21] we first introduce the logarithmic barrier 
function \x Y^=\ l°g x j t° "replace" the inequality constraint x > in the primal and then, by 
using Lagrangian duality theory, write down the first-order optimality conditions 

Ax = b, 

A T y + s -Qx = c, , , 

XSe = fie, { ' 

(x,s) > 0, 

where X and S are diagonal matrices in TZ nxn with elements of vectors x and s spread across 
the diagonal, respectively and e G TZ n is the vector of ones. This system of equations has a 
unique solution (x(fj,),y(fj,), s(jj,)), x(p) > 0, s(/i) > for any fj, > 0. The corresponding point is 
called a //-centre. A family of these points for all positive values of // determines a continuous 
curve {(x(fj,),y((j,),s(fj,)) : \x > 0} which is called the primal-dual central trajectory or central 
path. 
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The first two equations in ([2D are the primal and dual feasibility conditions, respectively. The 
third equation is a perturbed complementarity condition; the parameter \x associated with the 
logarithmic barrier function controls the level of perturbation. The convergence to optimality 
is forced by taking the barrier parameter \x to zero |21|, [TT] . 

In this paper we will assume that we work with the feasible interior point method and therefore 
all iterates (x,y,s) satisfy the first two equations in ([2]). Consequently, the duality gap is equal 
to the complementarity gap 



rp X '7 n , r p X rji . rri 

c x + —x Qx — (by — —x Qx) = x s 



(3) 



and by reducing the barrier parameter //, IPM achieves convergence to optimality. The stan- 
dard interior point algorithm applies the Newton method to ([2]), that is, computes the Newton 
direction and makes a step in this direction followed by a reduction of the barrier parameter [i. 
The reduction of the barrier term is enforced by the use of the parameter a € (0, 1) and setting 
^new _ a ^ j-j ue ^ ^ e f eas ibiiity of (x, y, s) the residual in ([2D takes the following form 

(b - Ax, c - A T y - s + Qx, a^e - XSe) = (0, 0, £). 



(4) 
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Hence the Newton direction (Ax, Ay, As) is obtained by solving the following system of linear 
equations 



(5) 



where I denotes the identity matrix of dimension n. 

In this paper we will analyse the method which allows the system ([5]) to be solved inexactly. To 
be precise, we will assume that all the iterates remain primal and dual feasible and an inexact 
Newton direction (Ax, Ay, As) satisfies the following system of linear equations 



(6) 



which admits an error r in the third equation. Let us observe that any step in such a primal-dual 
inexact Newton direction preserves primal and dual feasibility. 

By using the positive semidefiniteness of Q and exploiting the first two equations in the appro- 
priate Newton system, it is easy to demonstrate that for both the exact ([5]) and the inexact © 
Newton direction (Ax, Ay, As), the following property holds: 
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Ax T As = Ax T Q Ax > 0. 



(7) 



The third equation in the Newton system plays a crucial role in the convergence analysis of an 
interior point algorithm. For the inexact Newton direction ([6]) this equation takes the following 
form 



SAx + XAs = £ + r = cr/ie - XSe + r, 
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and by using e T e = n and i T i = n/i we get 

s Ax + x As = a\xe e — x s + e r = (a — l)x s + e r. 

Hence the complementarity gap at the new point (x(a),y(a), s(a)) = (x,y,s) + a(Ax, Ay, As) 
becomes 

x(a) T s(a) = (x + aAx) T (s + gAs) 

= x T s + a(s T Ai + x T As) + a 2 Ax T As 

= (1 - a(l - a))x T s + ae T r + a 2 Ax T As (8) 

and the corresponding average complementarity gap is 

fj,(a) = x(a) T s(at) / n = (1 — a(l — cr))n + ae T r/n + a 2 Ax T As/n. (9) 

Under the condition that the error term e T r and the second order term Ax T As are kept small 
enough in comparison with a(l — a)x T s, the complementarity gap at the new point is reduced 
compared with that at the previous iteration, thus guaranteeing progress of the algorithm. 

The convergence analysis of an interior point algorithm relies on imposing uniform progress 
in reducing the error in the complementarity conditions which technically is translated into a 
requirement that the error is small and bounded with 0(h). To achieve it we will restrict the 
iterates to remain in the neighbourhood of the central path {(x(fj,),y(fj,), s(fi)), fJ> > 0} and we 
will control the barrier reduction parameter a and the stepsize a so that x(a) T s(a) in ([8]) is 
noticeably smaller than x T s. 

We will consider two different ways of controlling the proximity to the central path and in both 
cases we will prove the convergence of the inexact interior point method and derive the worst- 
case complexity result. In both cases we will consider the feasible interior point algorithm and, 
hence, we will assume that all primal-dual iterates belong to the primal-dual strictly feasible 
set J 70 = {(x, y, s) | Ax = b, A T y + s — Qx = c, (x,s) > 0}. All iterates are confined to 
a neighbourhood of the central path which translates to a requirement that the error in the 
perturbed complementarity condition ([2]) is small. Depending on the norm used to measure this 
error, we will consider two different neighbourhoods: 

• a small neighbourhood induced by the use of the Euclidean norm for some 9 G (0,1) 

N 2 (9) = {(x, y, s) G T° | \\XSe - HI < M> (10) 
which yields a short-step algorithm, and 

• a symmetric neighbourhood induced by the use of the infinity norm for some 7 G (0, 1) 

N S (j) = {(x, y,s)eT°\ lf i< XjSj < -M, Vj}, (11) 
which yields a long-step algorithm. 

The former has a theoretical importance as it leads to the algorithm with the best complexity 
result known to date. However, it is not an implementable method because it leads to poor 
performance in practice. Indeed, its behaviour reproduces the worst-case analysis. The latter 
neighbourhood has a practical meaning and leads to an efficient algorithm in practice. 
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Following the general theory of the inexact Newton Method [T3], we will assume that the 
residual r in © satisfies 

Ikllp < 6\\Z\\ P , (12) 

for some 5 £ (0, 1) and an appropriate p-norm. According to the type of the neighbourhood 
used, (|10p or (jlip . this inequality will use either p = 2 or p = oo, respectively. Further in the 
paper we will omit a subscript 2 for the Euclidean norm unless an expression involves different 
norms at the same time and such an omission could lead to a confusion. 

In the next section we will prove that the feasible interior point algorithm using an inexact New- 
ton direction © and applied to a convex quadratic program converges to an e-accurate solution 
in £?(y / nln(l/e)) or C(nln(l/e)) iterations if it operates in the A^(0) or Ng(j) neighbourhood, 
respectively. Our analysis will follow the general scheme used by Wright [21J. 



3 Worst-case complexity results 



The analysis for two different neighbourhoods will share certain common features. The algorithm 
makes a step in the Newton direction obtained by solving ([6]). When a step in the Newton 
direction (Ax, Ay, As) is made, the new complementarity product for component j is given by 

Xj(a) sj(a) = (xj + aAxj)(sj + aAsj) 

= XjSj + a(sjAxj + XjAsj) + a 2 AxjAsj. (13) 

The third equation in © is a local linearization of the complementarity condition and controls 
the middle term SjAxj + XjAsj = £j + rj in the above equation. The error in the approximation 
of complementarity products is determined by the second-order term AxjAsj in (|13p . We will 
provide a bound on these products, namely, we will bound the vector of the second-order error 
terms HAXASeH. 

Having multiplied the third equation in the Newton system © by (XS)^ 1 ^ 2 , we obtain 

X-^S^Ax + X^S-^As = (XS)- 1 / 2 ^ + r). (14) 

Defining u = X- 1 / 2 S 1 l 2 Ax and v = X 1 / 2 S~ 1 l 2 As and using (0) we obtain u T v = Ax T As > 0. 
Let us partition all products UjVj into positive and negative ones: V = {j \ UjVj > 0} and 
■M. = {j | UjVj < 0} and observe that 

o < u T v = ^2 UjVj + ^2 UjVj = ^2 Mil - Yl I wi- ( 15 ) 

jeV jeM jev jeM 

Next, let us write equation (fTl"]) component-wise as Uj + Vj = {xjSj)~ l / 2 {^j + rj) for every 
j & {1, 2, . . . , n} and take the sum of squared equations for components j € V- 

< ^2(uj + Vj) 2 = Yitf + V 2 ) + 2 Y ^Vj = ^(xjsj)- 1 ^ + rj) 2 , 
jev jev jev jeV 

to get 

jev jev jev 
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Inequality (fT5j) implies YljeM \ u j v o\ — ^2jeV \ u 3 v j\ anc ^' nence > we can write 
||AXA5e||i = \ u j v j\ + Ki^il — 

n 

je^ i=i 

We will now consider two different algorithms: the short-step method in which the iterates 
are confined to N^{9) neighbourhood (jlOp and the long-step method in which the iterates are 
confined to Ngipf) neighbourhood (|lip . The names "short-step" and "long-step" describe the 
steps to optimality and are related to the choice of barrier reduction parameter a (and should 
not be confused with the stepsizes taken in the Newton direction). In the short-step method, a 
is very close to one and therefore the algorithm makes only a short step to optimality while in 
the long step method, a is usually a small number satisfying a<l. 



3.1 Analysis of the short-step method 

In this section we will assume that (x, y, s) G A^(^) for some 9 G (0, 1) and inequality (fl2l) holds 
for the 2-norm: [|r||2 < £||£||2- It is easy to deduce that since \\XSe— yue||oo < ||X5e — /ie[|2 < 9/j,, 
the complementarity products satisfy the following inequality 

(1 - 6)n < XjSj < (1 + 0)n Vj. (17) 

The barrier reduction parameter for the short-step algorithm is defined as a = l—/3/y/n for some 
/3 G (0, 1). Such a definition implies that (1 — a) 2 n = f3 2 and therefore, using e T (XSe — \xe) = 0, 
the norm of the term £ = o[ie — XSe in ([4]) satisfies 

HXSe - o[ief = \\{XSe - fie) + (1 - a)ne\\ 2 

= \\XSe - HI 2 + 2 (1- o) ne T (XSe-fj,e) + {l-a) 2 n 2 e T e 
< 6 2 fi 2 + (1 - a) 2 n[i 2 

= (0 2 + /3 2 )/i 2 - (18) 
We are now ready to derive a bound on ||AXA5e||. 

Lemma 3.1 Let 9 G (0, 1). If (x, y, s) G ^(0) then the inexact Newton direction (Ax, Ay, As) 
obtained by solving satisfies 

l|AA - ASe|| < (i±|i^!), (19) 



Proof: Inequality (|17|) provides a bound minj-fxjSj} > (1 — 6)fi. We use it to rewrite (|16|) : 

1 n 1 

||AXA Se || 2 < BAXASell, < _^V> + < __ K + r| > (20) 

Using (|12j) (with p = 2) and f)18|) we write 

I!? + r|| 2 < (IICII + ||r||) 2 < (1 + 5) 2 ||£|| 2 < (1 + S) 2 (9 2 + /3 V, 
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and after substituting this expression into ([2"Dj) obtain the required inequality (fTUj) . □ 

Next we will show that for appropriately chosen constants 9, j3 and 5, a full Newton step is feasible 
and the new iterate (x, y, s) = (x, y, s) + (Ax, Ay, As) also belongs to the N 2 (9) neighbourhood 
of the central path. We will prove an even stronger result which is that for any step a S (0, 1] 
in the Newton direction the following point 

(x(a), y{a), s{a)) = (x, y, s) + a(Ax, Ay, As) (21) 

is primal-dual feasible and belongs to the N2(9) neighbourhood. 

Using (fT3"]) and © and sjAxj+XjAsj = a^ — XjSj+rj, the deviation of the jth complementarity 
product from the average becomes 

Xj(a)sj(a) — n(a) = (xj + aAxj)(sj + aAsj) — /i(a) 

= XjSj + a(sjAxj + XjAsj) + a 2 AxjAsj — /j,(a) 

= (1— a)xjSj + aa[i+arj+a 2 AxjAsj —(l—a)fi — aafi—ae T r/n — a 2 Ax T As/n 



n) 



= (1 — a)(xjSj — fi) + a(rj — e r/n) + a (AxjAsj — Ax As/ 
Consequently, the proximity measure for the point (x(a), y(a), s(a)) becomes 

\\X(a)S(a)e-u(a)e\\ < (l-a)\\XSe- fie\\+a\\r- — e\\+a 2 \\AXASe ^— ^ell. (22) 

n n 

The Lemma below sets three parameters: the proximity constant 9 £ (0, 1) in (|10p . f3 G (0, 1) in 
the barrier reduction parameter a = 1 — fij^fn, and the level of error 5 6 (0, 1) allowed in the 
inexact Newton method (I12D. 



Lemma 3.2 Let (x,y,s) be the current iterate in A^(^) neighbourhood and (Ax, Ay, As) be the 
inexact Newton direction which solves equation system (0|). Let 9 = ft = 0.1. If 5 = 0.3 then for 
any a £ (0, 1] the new iterate \21\) after a step a in this direction satisfies 

\\X(a)S(a)e - fi(a)e\\ < 9n(a). (23) 
Proof: Expanding the square and using e T e = n we write 

n C^V no n n2 1 / T \2 T 2 T 9 

r e = r H — ^-(e r) e e (e r) 

n n z n 

= W 2 -i(e T r) 2 <||r|| 2 . (24) 
n 

Similarly, expanding the square, using e T e = n again and (AIA5e) T e = Ax T As, we write 

\\ AX ASe-^^e\\ 2 = ||AXASe|| 2 + ±-(Ax T As) 2 e T e - ^^(AXASefe 
n n z n 

= ||AXA5e|| 2 --(A2; T As) 2 < ||AXA5e|| 2 . (25) 
n 

Next, using ([22]), the definition of N 2 {9) and inequalities ([MD, {12]) and ([25]), we write 

\\X(a)S(a)e — /i(a)e|| < (1 — a)\\XSe— ne\\+ct\\r e||+Q 2 ||AXA5e ell 

n n 

< (l-a)9n+a5\\Z\\+a 2 \\AXASe\\. 
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Inequalities (|18p and (|19|) (Lemma 13. X j) provide bounds for the last two terms in the above 
inequality. We use them to write 



(1 + 5) (9 + B ) 

\X(ct)S(a)e-fi(a)e\\ < (l-a)9 l x+aS^9 2 + /3 2 /x+a L <Z V 

(1 — 9) 



2/a2 i »2\ 
: l+5) 2 (e 2 +/3 2 ) _ 2(l+<5) 2 



The choice of = 8 = 0.1 guarantees that yj9 2 + B 2 < \/29 and 1 ( \i ) j = ( 9 ; 6* hence 

||X(o!)5(a)e-At(a)e|| < (l-a)9fi+V2a59fi+a 2 2<yl ± ^ fl/x. 

Using equality ([9]) we observe that this lemma will be proved (inequality (|23p will be satisfied) 
if the following holds 

(l-a)9 f i+V2a59 f i+a 2 ^ 1 + S ^ 9fi < 9 ( (1 - ail - a))fi + a— + a 2 AxTAs ) , 

9 \ n n J 

We further simplify this inequality by removing the same terms present on both sides of it and 
then dividing it by a9. Inequality ([7]) guarantees that Ax T As is nonnegative, hence we conclude 
that (|23p will be satisfied if 

f=. 2(1 + <5) 2 ^ e T r 
9 n 

We observe that \e T r\ < ||e||||r|| = "v/n<5||£|| and hence, using (fl~8j) . we write 



' ' > • - 5 77, TT V259 



— I < -Z-Jp + pp < (26) 



Therefore to guarantee that (I23p holds for any a E (0, 1], it suffices to choose 5 such that 

r , 2(l + 5) 2 V259 

V2dfi H /i < an -^n 

9 y/n 

and this simplifies to 

Vn 9 Vn 

The left hand side of this inequality is an increasing function of 5 and we can easily check that 
the choice 5 = 0.3 gives O.3\/2(l + 0/\/ra) + 3.38/9 < 1-0/y/n which holds for 9 = B = 0.1 and 
any n > 2. □ 

Lemma [3.21 guarantees that for any a G (0,1] the new iterate (|2T1) also belongs to the iV^fl) 
neighbourhood of the central path. We will set a = 1 and take the full step in the Newton 
direction. Observe that the inexact Newton direction ([6]) allows the error r to appear only in 
the third equation which means that the direction (Ax, Ay, As) preserves the feasibility of primal 
and dual equality constraints. The new iterate is defined as (x, y, s) = (x, y, s) + (Ax, Ay, As) 
and setting a = 1 in gives 

. e T r Ax T As 

fj, = fj,(a) = apL-\ 1 . (27) 

n n 
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With 9 = f3 = 0.1 and 5 = 0.3 the right hand side term in inequality (|19|) may be simplified to 
give ||AXASe|| < 0.38/3^ and, using the Cauchy-Schwartz inequality, we get the bound 

Ax T As (AXASe) T e l llAVA „ n . , 0.38 „ 

< -| \AXASe\ e < — =)8/i. 



n n n ^Jn 

Using this inequality, our choice 9 = (3, ([26]) and a = 1 — f3/y/n, we obtain the following bound 
on p in (1271) 

<S . 25(3 0.38/3 n , 



where rj = /3(1 - 26 - 0.38) = 0.002. 

We are now ready to state the complexity result for the inexact short-step feasible interior point 
method operating in a A^O.l) neighbourhood. 

Theorem 3.1 Given e > 0, suppose that a feasible starting point (x°,y°,s°) £ A^O.l) satisfies 
(x°) T s° = n/jP, where fjP < l/e K , for some positive constant k. Then there exists an index L 
with L = 0(sjn ln(l/e)) such that fj < e, VZ > L. 

Proof: is a straightforward application of Theorem 3.2 in Wright \21\ Ch. 3]. □ 



3.2 Analysis of the long-step method 

In this section we will assume that (x,y,s) £ Ns(l) f° r some 7 £ (0,1) and inequality (fT2l) 
holds for the infinity norm: ||^||oo — ^||^||oo* ^Ve will ask for an aggressive reduction of the 
duality gap from one iteration to another and set the barrier reduction parameter a G (0, 1) to 
be significantly smaller than 1. Therefore it will not be possible in general to make the full step 
in the Newton direction. 

Using definition (jlip of the symmetric neighbourhood ^5(7) and observing that I/7 — 1 > 1 — 7 
we derive the following bound for the term £ = a fie — XSe in ([3]) 

\\XSe - CT/ueHoo = \\(XSe - fie) + (1 - a)fie\\oo 

< \\XSe — fie\\oo + (1 — a)fi 

< max{l — 7, — — l}fi + (1 — cr)fi 

7 

= (29) 

7 

We are now ready to derive a bound on || AXASeWoo. 

Lemma 3.3 Let 7 6 (0, 1). // (x, y, s) 6 Ns(j) then the inexact Newton direction (Ax, Ay, As) 
obtained by solving satisfies 

WAXASe^ < ||AJfA5e||i < _ ( 30 ) 



and 



Ax,A Sj <ii±^(i-a)V, Vj. (31) 
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Proof: The definition of the Ns("f) neighbourhood provides a bound mmj{xjSj} > 7/i. We use 
it to rewrite ([TBI): 



n 

HAXA.Selloo < ||AXA5e||i < — - f rVfe+rj) 2 < — U + r\\l (32) 

mmj{XjSj\ *r^y 7M 

Using (|12p (for the infinity norm) and (|29p we write 

11^ + r\\ 2 2 < nU + r||L < n(l + 5) 2 ||£||L < n(l + 5) 2 (i - a) V, 

7 



and, after substituting this expression into (|32l) . obtain the required inequality (1301) , We observe 
that (j30|) implies that 

+ \2 ^ a A <T (l±_g / 1 vy- 
— n ( g) u < Axj/lSj < n ( a) u, Vj, 

7 7 7 7 

but we can obtain a tighter upper bound for this component- wise error term. For this we write 
equation (fbi|) for component j and square both sides of it to get 

2Ax J A 8j < + Si! < (1 ± S ^ < Il±j)! ( l - g) y v A 

7M 7 7 

which implies (|3ip and completes the proof. □ 

Next we will show that for appropriately chosen constants a, 7 and (5, a (small) step a = 0(l/n) 
in the inexact Newton direction is feasible and the new iterate (x(a),y(a), s(a)) = (x,y,s) + 
a(Ax, Ay, As) remains in the Ns("f) neighbourhood of the central path. 

The Lemma below provides conditions which have to be met by three parameters: the proximity 
constant 7 6 (0,1) in (jlll) . the barrier reduction parameter a G (0,1), and the level of error 
5 £ (0, 1) allowed in the inexact Newton method (I12p . 

Lemma 3.4 Let (x, y, s) be the current iterate in the Ng(j) neighbourhood and (Ax, Ay, As) be 
the inexact Newton direction which solves equation system |2J)- // the stepsize a £ (0, 1] satisfies 
the following conditions: 

a{ 1 + n) il + 5) \ --a) 2 < a(l - 7) - 6(1 + 7 )(- - a) (33) 

7 7 7 

5(1 + -)(-- a) + a { -^±^(--af < (- - l)a (34) 

7 7 7 7 7 

i/ten t/ie neu? iterate (x(a),y(a), s(a)) belongs to the Ng(*y) neighbourhood, that is: 

7M(«) < Xj(a)sj(a) < ~/i(a), Vj. (35) 

Proof: Using the average complementarity gap © at the new iterate (x(a), y(a), s(a)) together 
with expression (fT3j) and the third equation in ([6]), we deduce that the left inequality in (J35j) 
will hold for any j if 

(e^V Ax^ As\ 
(1 — a)fi + ao[i + a V a 2 ) < (1 — a)xjSj + ao[i + ar 7 - + a 2 AxjAsj. 

n n ) 
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Since (x, y, s) E Nsfa) we know that 7(1 — oc)n < (1 — a)xjSj and therefore the above inequality 
will hold if we satisfy a tighter version of it: 

( N 2 Ax T As\ ^ /-I N 2A A 

7 I (1 — Qj/i + olct fi + a h a < 7(1 — a)n + acr^x + ar 7 + a Ax 7 A,s 7 . 

\ n n J 

After removing identical terms from both sides and dividing both sides by a, we conclude that 
the inequality will hold if 

(Ax T As \ e T r 
7 AxjAsj J < <r(l — 7)/u + rj — 7 . (36) 

Using Lemma 13.31 we deduce that 

= — < - \\AXASe 1 e U < -(- - afii (37) 

n n n 7 7 

hence 

Ax T As . . . ,(l + 5) 2 1 , 2 
7 AxjAsj <(~f + n) y —!- {-- of 11. (38) 

Using (|12p (for the infinity norm) and (129h we deduce that H^Hoo < <5||£||oo < — er)/i and 

|e T r| < ||r||i < n||r||oo < ^<5( a)fi (39) 

7 

hence 

| r . -7—I < |r,-| +7I — I < *(l + 7)(~ -°>- ( 40 ) 
n n 7 

The inequalities (|38|) and (|4U|) allow us to determine the most adverse conditions in (|36|) , namely 
when its left-hand-side is the largest possible and the right-hand-side is the smallest possible: 

a(7 + n) ^ 1 + ^ {--a-fn < a(l - 7)// - 6(1 + 7)(- - o)[i. 

7 7 '7 

We then conclude that (|33|) implies (|36[) and therefore the left inequality in (|35p holds. This 
completes the first part of the proof. 

The second part deals with the right inequality in (f35|) . Again, using Q, (fl"3j) and the third 
equation in Q, we write the required inequality which should be satisfied for any j G {1, 2, . . . , n} 

2 a a 1 //1 \ oAx T As 

(l — a)x 7 s 7 + acr/i + ax 7 + a Ax 7 As 7 < — (l — a.) a + aer/i + a h a 

7 \ n n 

and determine the condition under which it holds. We use similar arguments as in the earlier 
part of the proof: for example, (x,y,s) G ^5(7) implies (l — a)xjSj < ^(l — a)/i. Hence we 
simplify this inequality by removing identical terms and then divide both sides of it by a. The 
inequality we need to satisfy becomes 

e T r ( Ax T As\ l 

r 7 - V a AxnAsn < ( lW/i. (41) 

jn \ l n J 7 
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Using (|3"Tj) and ([7]) we write 

Ax,- As,- < -(- - of\x (42) 

-yn 7 7 

and using (|12p and (j29j) and similar arguments to those which led to (|40p we write 

p-^V p^V I I 

\Tj ~ — | < hi + | — | < (5(1 + -)(- - a)n. (43) 

7n 7n 7 7 



By (|42j) and (|43|) . to satisfy ([41]) we need to choose a such that: 

Iwl x (1 + 5) 2 , 1 , 9 ,1 
5 1 + -)(-- + a ~ " V < - " 1^ 

7 7 7 7 7 



which simplifies to ([34]) and completes the second part of the proof. □ 

Lemma 13.41 provides conditions which the stepsize a £ (0, 1] needs to satisfy so that the new 
iterate remains in the ^5(7) neighbourhood of the central path. We still need to demonstrate 
that after a step is made a sufficient reduction of duality gap is achieved. 

Lemma 3.5 Let (x, y, s) G Ns("f) be given and let (Ax, Ay, As) be the inexact Newton direction 
which solves equation system (0). // the stepsize a 6 (0, 1] satisfies the inequality 

a + 6(~ - a) + a (1 + ^ (- - af < 0.9, (44) 

7 7 7 

then the duality gap at the new iterate (x(a),y(a), s(a)) satisfies: 

(i(a) < (1 - 0.1a)/i. (45) 

Proof: By substituting ([9]) into (I45p . cancelling similar terms and dividing the resulting in- 
equality by a, we replace (|45p with a new condition that the stepsize a has to satisfy: 

e T r Ax T As 
a/1 H h a < 0.9/i. 

n n 

Using the bounds ([39]) and ([37|) derived earlier we conclude that the above inequality will hold 
if 

a\i + 5(- - a)n + a ( 1 + ^ (1 _ o-) 2 ^ < o.9/i, 

7 ' 7 7 



which is equivalent to (|44p . D 



It remains to consider the three conditions (|33|). (|34|) and (|44p and to demonstrate that an 
appropriate choice of parameters 7, a and 5 guarantees that all these conditions hold for some 



We set the proximity constant 7 = 0.5 in (jlip . the barrier reduction parameter a = 0.5 and 
5 = 0.05 as the level of error allowed in the inexact Newton method (|12p . Indeed, with these 
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parameter settings (^ — a) 2 = 4.96125, and we verify that all three conditions (|33|) . (|34"|) 

and ()44p are satisfied by a = for any n > 2. Substituting such an a into (|45p gives 

p = /i(a) < (1 

n 

where 77 = 0.002, and allows us to conclude this section with the following complexity result for 
the long-step inexact feasible interior point method operating in a Ns (0.5) neighbourhood. 

Theorem 3.2 Given e > 0, suppose that a feasible starting point (x° ,y° , s°) G Ns (0.5) satisfies 
(x°) T s° = n/jP, where < l/e K , for some positive constant k. Then there exists an index L 
with L = 0(n ln(l/e)) such that ft 1 < e, VZ > L. 

Proof: is a straightforward application of Theorem 3.2 in Wright \21\ Ch. 3]. □ 



4 Practical aspects of the inexact IPM 

Linear system (|6|) is a modification of ([5]) which admits the error r in its third equation. The 
analysis in the previous section provides guarantees that when the Newton equation system ([6]) 
is solved inexactly and error r satisfies condition (|12p the interior point algorithms retain their 
good complexity results. In this section we will briefly comment on a practical way of computing 
the Newton direction which meets condition (1121). 



(46) 



Let us observe that after eliminating As, ([5]) is reduced to the augmented form 

- _q _ e-i a t 

A 





' Ax ' 








Ay _ 








where = XS^ 1 G TZ nxn . Any IPM has to solve at least one such system at each iteration 
[8j [H]. Numerous attempts have been made during the last decade to employ an iterative 
method for this task. Iterative methods for linear algebra are particularly attractive if they can 
be used to find only an approximate solution of the linear system, that is, when their run can be 
truncated to merely a few iterations. It is common to interrupt the iterative process once the 
required (loose) accuracy of the solution is obtained. Clearly such a solution is inexact and in 
the context of (I46D this translates to dealing with an inexact Newton direction (Ax, Ay) which 
satisfies 



-Q-e- 1 

A 



A T 




Ax 
Ay 



(47) 



where the errors r T G lZ n and r,, G lZ m determine the level of inexactness. 



The analysis presented in this paper applies to the situation when r y = 0. The other error, r x 



may take a nonzero value and indeed, ()47j) becomess equivalent to 



if r v = and 



X-^ + r x 



(48) 



This equation combined with condition (I12D determines practical stopping criteria set for an 
iterative solution method applied to (|4"6j) : 



and 



\Xr x \\<5U\\- 



(49) 
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Interestingly, there exist classes of iterative methods which can meet the above stopping criteria. 
They belong to a broad collection of iterative methods for saddle point problems [6] and rely on 
specially designed indefinite preconditioners [T5J ED] ■ 

5 Conclusions 

The analysis presented in this paper provides the proofs of 0{yfn log(l/e)) and C(nlog(l/e)) 
iteration complexity of the, respectively, short-step and long-step inexact feasible primal-dual 
algorithms for quadratic programming. The analysis allows for considerable relative errors in 
the Newton direction. Indeed, 5 in (I12D may take values 0.3 and 0.05 for the short-step and 
long-step algorithms, respectively. This shows, somewhat surprisingly, that the inexactness in 
the solution of the Newton equation system (J6| may be quite considerable without adversly 
affecting the best known worst-case iteration complexity results of these algorithms. It is an 
encouraging result for researchers who design preconditioners for iterative methods and wish 
to apply them to solve the reduced Newton equation systems arising in the context of interior 
point methods. 
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